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(54) Title: DATAGRAM TRANSMISSION OVER VIRTUAL CIRCUITS 
(57) Abstract 

A method and apparatus for an enhanced datagiam packet 
switched computer networic is disclosed. Hie invention processes 
network datapam packets in netwoik devices as separate flows, 
based cm the source-destination address pair contained in die 
datagram packet itself. As a result, the netwotk can control and 
manage each flow of datagrams in a segregated fashion. The 
processing steps that can be specified for eadi flow include traffic 
management, flow control, packet forwarding, access control and 
other network management functions. The abflity to control 
network traffic on aper flow l»sis allows for the efficient handling 
of a wide range and a large variety of network traffic, as is typical 
in large-scale computer networks, including video and multimedia 
type traffic The unount of buffer resources and bandwidth 
resources assigned to each flow can be tndividimUy controlled by 
netwoik management. In the dynamic operation of the network, 
these resources can be varied t»sed on actual network traffic 
loading and congestion encountered. Hie invention also teaches 
an enhanced network access control mediod which can selectively 
control flows of datagram packets entering the netwoik and 
travding between network nodes. This new network access 
control method also mteroperates with existing media access 
control protoccris, such as used in the Ediefnet or 8023 local 
area network. An important aspect of the invention is that it does 
not require any changes to existing network protocols or network 
applications. This is accomplished by specify tng the flow omtiol, 
traffic management and netwoik control functions via netwcHk 
management. Applications or network protocols that require the 
netwoik to provide a certain level of bandwidth or performance 
can request such services via the netwoik management function. 
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DATAGRAM TRANSMISSION OVER VIRTUAL CIRCUITS 



Held of the Invention 

The present invention relates to the field of computer networks. More 
particularly, the present invention relates to the field of computer networks which 
S are based on datagram packet switching. 

Background of the Inv^tion 

Compute Networks are used to interconnect computers and peripherals to 
10 allow them to exchange and share data such as files, electronic mail, databases, 
multimediaAddeo, and other data. 

Packet Switching 

15 Nearly all computer networks use packet switching, which divides longer 

units of data transfer mto smaller packets which are sent separately over the 
netw<»rk. This allows each packet to be processed independently from another 
packet without having to wait for tte entire data transfer to be ccxnpleted. It also 
enables conmiunications between a plurality of computer systems to be intermixed 

20 on one network. Host interfaces connect the computer systems to a network 
allowing each computer system to act as the source and destination of packets on 
the network. 

A first key issue in packet switched networks is addressing. The addressing in 
25 packet switched networks is conventiraally performed by one of two approaches, 
known as virtual circuit packet switching or datagram packet switching. 

Virtual Circuit Packet Switching 

30 In the virtual circuit approach^ before any data can be transmitted, a virtual circuit 
must be first established along the patfi from the source to the destination in advance 
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of any communication. After the virtual circuit is setup, the source can then send 
packets to the destination. Each packet in the virtual circuit approach has a virtual 
circuit identifier, which is used to switch the packet along the path from source to 
the destination. 

5 

The virtual circuit apix^>ach reduces the size of the identification required in each 
packet header. It also allows additional information about the packet handling to be 
established as part of the virtual circuit setup operation. Another claimed benefit- is 
that forwarding and switching of virtual circuit padcets can be made more efficimt 

10 because of die virtual setup process. However, the virtual circuit 2q;)proach incurs 
the cost of delay to setup Ae virtual circuit befoie sending any data, and it incurs the 
cost of maintaining the virtual circuit state in each network device along die virtual 
circuit padi, even if a virtual circuit is idle. Also, in practice the memory space for 
virtual circuit state in network devices has limited the number of circuits that are 

IS available, which cont^licates die behavior of netwcHk nodes that need to create 
virtual circuits to c(Hiununicate. 

Datagram Packet Switching 

20 In the datagram approach, each datagram packet is a self-contained unitof 

data delivery. A typical datagram packet includes a globally unique source address, 
a globally unique destination address, a (MPOtocol type field, a data field, and a cyclic 
redundancy checksum rCRCT) to insurc data integrity. 

25 Datagrams can be sent without prior arrangement with the network, i.e. 

without setting up a virtual circuit or ccmnecticm. Each network device receiving a 
ds^^si^m packet examines the destinaticm address included in the datagram packet 
and makes a local decision whether to accept, ignore, or forward this packet 

30 Various conventional network devices learn informaticm from observing 

datagram packet traffic in data networks. For example, a conventional network 
switch device that interconnects multiple network segments can "leam" the 
location of network statiras connected to its ports by monitoring the source address 
of packets received on its ports. After it has associated a station address with a 

35 certain port, the network switch can then forward datagram packets addressed to 
that station to that port In this type of device, the datagram source address is used 
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to Icam Ac location of a station on tfie netwoik, whereas the f(»rwaiding decision is 
made on basis of the datagram destination address alone. 

Datagram packet switching has the advantage that it avoids the overhead 
5 and cost of setting up virtual circuit connection in network devices. However, it 
incurs Ae expense of transmitting a larger packet header than required for virtual 
circuit switching, and it incurs the cost for processing this larger packet header in 
every network device to which it is deliv^ed. Also, there is no virtual circuit setup 
process to establish additional information for datagram packet im>cessing. Anodic 
10 disadvantage of datagram packet switching is that it is difficult to control packet 
flow to the same degree as with virtual circuits because there is, in the conventicmal 
case, no state in the netwoik devices associated with the trafiBc flow. 

The datagram packet switching approach has bcCT extensively used in shared 
IS media local area networks. Shared media netw«ks provide for a multiplicity of 
stations directly connected to the network, with die shared media i»oviding direct 
access from any transmitter to any receiver. Since the receives need to be able to 
distinguish pactets addressed specifically to them, each receiver needs to have a 
unique address. In addition, since the unit of access to the shared medium is one 
20 packet, each packet needs to contain the unique address capable to identify the 
receivCT. As a result, all commonly used local area networks are based on datagram 
packet switching and have no provisions for virtual circuit setup. 

Media Access Control Protocol 

25 

The network access mechanism in shared media local area network will now 
be further described* This function, commonly known as the media access contrcd or 
MAC protocol, defines bow to arbitrate access among multiple stations that desire to 
use the network. Individual staticms connected to the network have to adhere to the 
30 MAC protocol in order to allow pmpex netwcnk operaticm. 

A number of different media access control protocols exist The MAC 
protocol, in conjuncticHi widi die exact packet format, is the essence <^ what defines 
a local area network standard. The following is a brief overview of local area 
35 network standards that are in wide use today. 
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4 

The most widely used lcx:al area netwcxk is commooly known as Ethernet and 
employs an access protocol referred to as Carrier Sense Multiple Access with 
ColUsion Detection (CSMA/CD). [see U.S, Patent 4,063;220, issued Dec 13. 1977, 
for a Multipoint Data Communication Syst^ with Collision Detection, InventCM^ 
5 Metcalfe, Boggs, Thacker, and Lampson]. The current definition of the Ethonet 
CSMA-/CD protocol is defined in IEEE Standard 8023, published by the Institute 
of Electrical and Electronics Engineers, 345 East 45di Street, New York, NY 10017. 
The Ethernet standard specifies a data transmission rate of 10 or 100 
Megabits/second. 

10 

Another widely used local area network standard is Tokenring, also known as 
IEEE Std. 802^, transnoitting at a speed of 4 or 16 Mbits/sec and FDDI, or Fiber- 
Distributed-Data-Interface, which sends data at a speed of 100 Mbits/sec. Both 
Tokenring and FDDI are based on a circulating token granting access to the 
IS network, although their respective datagram packet formats and other operating 
aspects are unique to each standard. 

What is conuncm to all these media access control mechanisms is that they do 
not include provisicms for virtual circuit setup and have no provisions to specify 
20 attributes tfiat relate to virtual circuits, such as traffic management or flow control for 
specific ccmnections. This limits the ability of conventional local area netwwks to 
acconunodate higher level network functions or to support virtual connection 
oriented traffic mechanisms. 

25 Devices for Interconnecting Local Area Netwoiks Another key issue with 

datagram packet switched networics is how to intercoimect individual network 
segments into larger networks. The size and usage of datagram packet switched 
networks has grown nuich beycmd what was envisioned when these networks were 
designed. Devices such as bridges, switches, and routers, have been used to 

30 interconnect individual LAN segments into larger networks, but each have their own 
set of pr<4)l^ns in scaling to higher perfcnmance. 

Bridges forward datagram packets between network segments by learning 
the location of the devices on the network by observing the source address 
35 contained in datagram packets passing by. once the bridge has learned which 
network device ia located on which network segment, it can then fcMward datagram 
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5 

packets addressed to that netwcfk device to the impropriate networic segment one 
of die limitations of bridges is that they do not filtCT traffic beyond the data link 
level. 

S Switches are basically multi-port networic bridges that can forward datagram 

packets among a multiplicity of network ports. Frequently^ switches provide 
additional capabilities for assisting with network management^ including trafRc 
filtering and segmenting networks into virtual LANS. As in the case of bridges, 
switches have to fraward broadcasts to all prats configured into one virtual LAN. In 

10 addition, conventional switches cannot i^ovide fair service or priority service to 
individual traffic flows, and tiiey require significant amount of memory to avoid 
dropping packets in the case of network congestion. 

Routers also interconnect several network segments, but they operate 
IS primarily at the network protocol layff, rather than at the datagram packet layer. 
Routers participate in all netwcnic protocol functions, which traditionally requires 
general purpose (Hocessing. As a result, traditional routers are more expensive and 
have less throughput than switches. -In addition they are more difficult to 
administer. 

20 

Finally, virtual circuit packet switched networks, in particular ATM, have- 
been iHoposed to interconnect local area networic segments. However, it has turned 
out to be very difficult to map existing network protocols that are based on 
datagram packets to the ATM network architecture. 

25 

In summary, bridges and switches transparently extend the domain of 
networks, and allow for cost-effective and high-performance implementations. 
However, they cannot segment a netwofk effectively in terms of traffic manag«n»t 
and broadcast traffic. Routers, on the otiier hand, can segment networks very 
30 effectively, but are much more expensive and are performance bottieneck in high- 
speed netwcMfks. ATM has been very difficult to map to current network protocols. 

The ideal network device for interccmnecting network segments would have 
the high-speed and cost-effectiveness of a switch, with the ability of segment and 
35 manage network traffic similar to a router. 
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Traffic Management 

Another key issue in packet switched netwwks is traffic control or traffic 
management 

5 

In a packet switched network* each link at every switching node in the 
network represents a queue. As the traffic arrival rate at a link approaches its 
transmission rate, the queue length grows dramaticaUy and the data in the queue 
needs to be stored in die attached network nodes. Eventually, a network node will 
10 run out of packet buffer ci^aci^ which will cause further packets arriving to be 
discarded or dropped I>r<^q)ed packets are eventually r^ransmitted by the source, 
causing the traffic load to increase further. Evratually, the network can reach a state 
where most of the packets in the network are letransmissicms. 

IS Conventionally, two types of traffic control mechanism are used in packet 

switched netwofks: flow control and congestion control. Flow control is concerned 
with matching the transmission rate of a source station to die reception rate of a 
destination station. A typical networks flow control mechanism uses a window 
techniques to limit the number of packets a source can transmit which are not yet 

20 confirmed as having been received by the destinatic»i. Conventional flow control is 
an end-to-end mechanism that exists in certain network protocols, in particular 
connection oriented network protocols such as TCP/IP. However, conventional 
flow control between source and destination does not solve the network congestion 
I^oblem, since it does not take the utilization of buffer resources within the netw<»fk 

25 into account In addition, ncm-cormectien oriented network protocols do not use 
window based flow control. Also, continuous rate traffic sources such as real-time 
video don't match the nature of destinaticHi controlled behavior since the 
transmission rate is determined by the source. 

30 Problem Statraient 

What is needed is an improved method and apparatus for high-speed 
datagram packet switched networics that can support a large number of network 
staticms, a wide range of network transmission speeds, a wide variety of source 
35 traffic bdiavi w including video and multimedia, while maintaining conq^atibility with 
existing netw<»k protocols and s^Iications. 
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SUMMARY OFTHE INVENTION 



Methods and apparatus for an enhanced datagram packet switched conq>uter 
network are disclosed. 

5 

The invention processes network datagram packets in network devices 
as.separate flows, based on the source-destination address pair contained in the 
Hfttflgrflm packet itself. As a result, the network can control and manage each flow 
of datagrams in a segregated £ashi<m. The jMrocessing steps tiiat can be specified for 
10 each flow include traffic management, flow control, packet forwarding, access 
control, and oAer network managemrat fimctions* 

The ability to control network traffic on a per flow basis allows for die 
efficient handling of a wide range and a large variety of network trafific, as is typical 
15 in large-scale computer n^w<xks, including video and multimedia type traffic. 

The amount of buff^ resources and bandwidth resources assigned to each 
flow can be individually controlled by network management In the dynamic 
operation of the network, these resources can be varied based on actual network 
20 traffic loading and congestion encountered. 

The invration also includes an enhanced network access control method 
which can selectively control flows of datagram packets entmng the network and 
traveling between network nodes. This new network access control method 
25 interc^rates with existing media access control protocols, such as used in the 
Ethernet or 8023 local area netwodc 

An important aspect of the invention is that it can be implemented in network 
switching devices at very high performance and at low cost. High performance is 
30 required to match the transmission speed of datagram packets on the network. Low 
cost is essential such that it is economical to use the invention widely. 

In the prefmed implementation, both high-performance and low cost is 
achieved by partitioning the task of datagram flow processing between dedicated 
35 network switch hardware-and dedicated network switch software tiiat executes on 
a high-speed controller CPU. 
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8 

The netwoik switch hardware provides a multiplicity of network ports, a 
shared memory buffer for storing datagram packets, a virtual path cache that stores 
the state and processing instructions specific to the active datagram packet flows. 

5 

Datagram packets received on an input port are buffered in the shared 
memory buffer. The source-destination address pair in the datagram packet header is 
used to index die virtual palfa cache to find a matching entry. If a matching entry is 
found in the virtual path cache, then the switch hardware paforms all the packet 
10 processing steps indicated in the virtual path record, including traffic management 
and packet routing. 

If no matching entry is f<Hiiid in the virtual path cache, then the datagram 
packet is forwarded to the controller CPU for g»eral purpose processing. The 
IS controller CPU determines, through networic management data structures and 
software, how to process further datagram packets with this source-destination 
address in the switch hardware. The ccmtcoller CPU then loads an impropriate mtry 
into the virtual path cache. If all entries in the virtual path cache are in use, then the 
CPU n^oves the least recratly used entry before loading the new entry. 

20 

Brief Description of the Drawings 

Other features and advantages of die present invention will become more 
apparent to those skilled in the art from the following detailed description in 
25 conjunction with the ai^nded drawings in which: 

Hgure 1 is a block diagram of a computer ccxnmunication system; 

Figure 2 is a block diagram of the Eth^et datagram packet format; 

30 

Hgure 3 illustrates the Virtual Path Record data stmcture; 
Hgure 4 is a block diagram of the network switching device; 
35 Hgure 5 is a flow diagram of the network switching device operation; 
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Figure 6 is a block diagram of the virtual path cache; 
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Figure 7 is a block diagram of the virtual path hash function; and 
S Figure 8 is a block diagram of the transmit data structure. 

Detailed Description of the Invention 

An enhanced ccxnputer network communication system is disclosed. 

10 

To help understand the inventicm, the following definitions are used: 

A "datagram packet" is a self-contained unit of packet delivery on a packet 
switched network, which includes a destinaticm address field, a source address field, 
15 an opti<Hial type field, and a data field. 

The "destinaticMi address" and the ''source address" refer to the physical 
network device addresses contained in a datagram packet, both of which are unique 
within a network. 

20 

A "flow" is a plurality of datagram packets each packet containing an 
identical source-destination address pair. 

A "virtual path" is the communication path from a source to a destination in a 
25 datagram packet switched network. 

In the following desoiption, far purposes of explanation, specific numbm, 
times, signals, and other parameters are set forth in order to provide a thorough 
understanding of flie present invention. However, it vdll be apparent to anyone 
30 skilled in the art that die present invention may be (vacticed without these specific 
details. In other instances, well known circuits and devices are shown in block 
diagram in ord&r not to obscure the present invention urmecessarily. 

The datagram packet switched comnmnication system is illustrated in Hgure 1 
35 by way of four network switching devices 101, 102, 103, and 104 interconnected 
with each other via backbone links 105, 106, 107, and 108. In addition, switch 103 is 
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interconnected via links 110 through 112 to client computers 120 through 122, 
switch 104 is interconnected via links 113 through 115 to client computers 123 to 
125 and switch 102 is interconnected via links 116 and 117 to server computers 126 
and 127. 

5 

The netwcxk shown in Figure 1 is small for purposes of illustration. In practice 
most networks would include a much larger number of host computers and netw(»'k 
switching devices. 

10 The basic unit of transmissicHi and processing in this network is a datagram. 

¥ot purposes of Illustration, we will be using the Ethernet datagram packet format in 
this description, it will be af^arent to anyone skilled in tfie art that other datagram 
packet formats can also be used to practice this invention, including the different 
datagrams described in the IEEE 802 family of network standards. 

15 

As illustrated in Figure 2, the Ethernet datagram 200 contains a 48-bit 
destination address field 201, a 48-bit source address field 202, a 16-bit type field 
203, a variable size data field 204 ranging from 46 bytes to 1500 bytes, and a 32-bit 
CRC field 205. 

20 

A key aspect of die present invention is the virtual path method. A virtual 
path is specified by the source-destination address pair contained in a datagram 
packet In the Ethmet packet datagram, the source-destinatiiHi address pair can be 
thought of as a %-bit circuit idratifier, which specifies a unidirectional circuit from 

25 the source to the destination. This 96-bit circuit identifi^ will subsequently be 
referred to as a virtual path in order to avoid confusion with virtual circuit networics. 
While this 96-bit virtual path identifier may i^pear large as compared to the much 
smaller circuit identifim in virtual circuit networks, it has the significant advantage 
that it is globally unique and thus does not require to be mapped to different 

30 identifim as a datagram packet travels along the path from source to destination. 

Virtual Path Recml 

Each datagram packet arriving at a networic switching device is recognized as 
35 traveling on a particular virtual path by the source-destination address pair 
contained in this heada. The netwoik switch maps this source-destination pair to a 
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virtual path lecoid in the switch. The virtual path record specifies how the datagram 
packet is to be processed* including its routing, priority, scheduling and buffer 
resource aUocation. 

5 Hgure 3 illustrates the data structure of the virtual path record. 

It will be apparent to those skilled in the art that other data structures from the 
one ^own can be successfully used, including but not limited to fields of different 
size, arranging die fields in different order, and additional fields not present in 
10 Figure 3. 

Turning now to the specific virtual path record illustrated in Figure 3, there 
are four groups of fields: the tag field 310, the forwarding field 320, the state field 
330, and the statistics field 340. The function and meaning of these fields will now 
15 be described. 

TagFiekl 

The purpose of tiie tag field is to match an incoming datagram packet against 
20 a virtual path record. The tag field 310 has four subfields: the destination address 
field 311, the scMirce address field 312, the opticmal type field 313, and the input port 
field 314. 

Since the virtual path index is quite large, 96 bits in the case of Ethernet 
25 datagrams, it is not practical to provide a full array indexed by the virtual path 
number. Instead, each virtual path record is keyed, with the virtual path number to 
allow lookup by search or partial index. One method for organizing the virtual path 
recwds and looking them up will be furth^ described below. 

30 For the lookup method to locate the correct virtual path record, the 

destination address field 311 and the source address field 312 in the virtual path 
record must match the destinaticm address field 201 and the source address field 202 
in the datagram heada. 

35 Type field 313 allows for optional type filtering. If the type field is set to 0, 

any type field 203 in the datagram header will match this virtual path record. 
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However, if the type field is not 0, then the type field has to match the type field 203 
in the datagram header exactly for the match to be successful. 

The input port field 314 allows input port filtering. The input port field has to 
5 match the actual input port number at which the datagram packet has been received. 

Forwarding Field 

The fwwarding field 320 determines how the datagram packet should be 
10 fc^arded. Ou^ut port field 321 specifies die output port on which ttiis datagram 
will be tranmutted. FrioiiQr field 312 specifies the traffic managemrat priority <tf this 
virtual path compared to other virtual paths. Real time field 323 forces the switch to 
process this packet in real time mode» which includes die act of drof^ing the packet 
if it cannot be sent within a certain time. 

15 

Store Forward field 324 selects the store-forward mode of operation. 
NcMinally the switch operates in cut-through mode where an incoming datagram is 
sent on to the ou^ut port as soon as feasible, even before it is completely received. 
In store-forwaid mode, an incoming packet must be completely received before it is 
20 sent This is a requirramt in case of speed conversion from a slowa input port to a 
fast^ output port Store^fc»ward mode is also used to insure die conectness of the 
complete packet received before sending it on. 

Multicast field 325 selects the multicast mode of operaticui. Multicast mode 
25 involves scheduling of a single datagram packet on multiple ou^uts, which are 
determined by a bit vector in the output port field, with "1** bits indicating ou^t 
ports to which the Multicast should be sent 

Field 326 is the Snoop Mode. Sno<^ mode when selected sends a c<^y of the 
30 datagram packet to the CPU for graeral purpose processing. 

Field 327 is the Buffer Size field, which specifies the maximum number of 
packet buffers allocated to this path for buffering purposes. 

35 The state field 330, includes the following fields; Head Pointer Field 331, 

which points to the beginning of the buffer area associated with this virtual path; 
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Taa Pointer 332 points to the end of the buffer area of this virtual path; Uplink 
Pointer 333 points to the next virtual path record to transmit to the source, downlink 
pointer 334 points to.the next virtual path record to transmit to the destination. 

S The statistics field 340 maintains trafiic statistics regarding the traffic received 

oa this virtual path. Field 341 counts Ae number of packets received and field 342 
counts the number of bytes received on tfiis virtual path. 

Hgure 4 Preferred Inqdemaitation 

10 

Referring to Figure 4, a virtual path network switching device 400 is 
illustrated with Network Input Ports 401 through 404» netwcnk ou^ut ports 405 
through 408, switch hardware 409, whwe shared buffw m^nory 410, controU^ CPU 
411, CPU interface 412, CPU read-and-write memory 413, Hash PROM 414, and 
15 virtual path cache 415. Using Figure 4 and the flow chart in Figure 5, a cycle of 
<^)eTation will be described. 

Datagram packets arriving through network ports 401 through 404 are 
temp«arily stored in shared buffer memory 410. As soon as the datagram packet 

20 head^ has arrived, which in the case of Ethernet datagrams is after the first 14 bytes 
of the datagram packet, the virtual path cache is looked up to check whether a 
virtual path cache entry exists for this padu If a matching entry is found in the virtual 
patfi cache 415, tfien switch hardware 409 starts processing the datagram packet as 
specified in the virtual path cache entry which in the typical case will f(Mwanl the 

25 Hat^gram packet ou oue oi the ou^ut ports 405 through 408. 

If no entry matching die datagram was found in the virtual path cache 415, 
then the datagram packet is forwarded to controller CPU 41 1 via CPU interface 412 
for general purpose processing. Controller CPU 411 processes datagram packet 
30 according to instructions and data stored in main memory 413 and optionally in 
Flash PROM 414. Said instructions and data structures used for datagram 
processing have been created previously by network management, network 
configuration, networic statistics, and netwoik behavior. 



35 



The result of the datagram general purpose processing is that the CPU 
determines how future datagram pactets on this virtual path should be processed by 
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the switch hardware 409 and loads an i4>propriate entry into the virtual path cache 
415* If all entries in the virtual path cache 415 are in use, ibcn controller CPU 409 
removes the least recently used entry in virtual path cache 415 before loading the 
new entry. CPU 41 1 then forwards the datagram packet to the switch hardware 409 
via CPU interface 412 for transmission. 

When the controller CPU loads a new virtual path cache entry, it sets the tag 
fkld 310 to the desired virtual path index, the forwarding field 320 to the desired 
forwarding function, and it initializes the state field 330 and the statistics field 340. 
The switch hardware will then automatically update the state and the statistics fields 
as tiie path is used. The switch hardware does not modify the information in the tag 
field and the forwarding field. 

Figure 5 How Diagram 

A method in accordance with this invention is shown in the flow diagram of 
KgureS. 

1. Packet Arrives at die invented systemu 

2. Check Datagram Virtual Patii Index against Virtual Patii Cache. 

3. If valid virtual path »try is found, process padcet in Switch Hardware 
which includes the steps of forwarding the datagram packet to 
ou^ut port and updating the statistics field 340. 

4. If No Entry is found, forward packet to Controller CPU to process 
datagram packet in software. Controller CPU sends datagram 
packet back to switch hardware for transmission on the a^xropriate 
output port. 

5. If Controller CPU determines that future datagram packets of this 
virtual path should be processed by switch hardware. Controller 
CPU then creates new virtual path rec<»d and writes it into virtual path 
cache, replacing the least recentiy used entiy if necessary. 

Virtual Path Cache organization 

Figure 6 illustrates an example of virtual path cache organization. 
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It will be apparent to anyone skilled in the art that other cache sizes and 
organizations from the one shown can be successfully used, including but not 
limited to caches of different size, associativity, alternative hash-function, and 
cmtent-addressability. 

5 

The virtual path cache illustrated in Figure 6 is organized as a 4-set 
associative cache buUt with four banks of high-speed static memory 601 thrcnigh 
604, each equipped with a set of comparators 611 ..614 that control a set of tri-state 
buffers 621 through 624. 

10 

The virtual path index 630, which is die source-destination address pair of the 
inc(xning datagram, enters hash function 631 which in turn produces a virtual patch 
cache index 632 which in turn looks up the four parallel sets of the virtual path 
cache SRAms 601 through 604. The tag field 310 from each set of SRAMs will be 
IS con^aied against the virtual padi index 630 and <»ly that virtual path record that 
matches will be ou^ut on the virtual path record databus 633 via tri-state diivos 
621 through 624. Combinatorial logic 634 will generate a high signal 635 to 
indicate a hit 

20 If no tag field matches, dien combinatMial logic 634 will generate a low signal 

635 to indicate a miss; i.e. tiiat no valid virtual path record was found in die virtual 
path cache. 

Hash Function 

25 

Figure 7 illustrates a specific hash functitHi raibodied in hash function logic 
631. Again, this hash function is used for illustration only. It will be apparrat to 
anyone sldlled in the art diat other hash functions from the one shown can be used. 

30 Refming now to Figure 7, die specific hash function logic is the bitwise 

Exclusive-OR 703 between die low-order 15 bits of die destination address 701 and 
die source address 702 of die Virtual Padi Index 630, producing the 15 bit virtual 
path cache index 632. A bit-wise Exclusive-OR functim is used because it is sunple 
and fast to in^lonent The low-order address bits are used from both swrce and 

35 destination address since they change widi the highest frequency. 
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Figure 8 shows a data structure diat illustrates how an output port transmits 
datagram packets buffered in die switch shared memory. This particular structure 
5 design is for purposes of illustration only; it will be apparent to anyone skilled in the 
art that odier data structures from the one shown can be used successfully. 

The data structure in Figure 8 is for one ou^ut port only. Referring briefly to 
Hgme 4, each output port 405 through 408 in the switch hardware 409 has a similar 
10 data structure to that shown in Figure 8. 

Output port 801 is to transmit datagrams buffered and waiting for 
transmission on virtual paths 810-1, 810-2 through 810-n, where n is a selected 
integer. 

15 

The ou^ut port 801 has a head pointer 802 and current pointer 803. Head 
pointer 802 points to tfie first entry 810-1 in the transmit list 804, which links to the 
next entry 810-2. Current pointer 803 points to the entry from which datagrams are 
to be transmitted next, which is virtual path 810-2 in tfiis exanq)le. Hie transmit list 

20 804 is formed by the link fields 811-1 through 811-n. Each link field points to the 
next padi in die transmit list The last link entry 811-n in the transmit list 804 has a 
link field value of 0. The actual length of the transmit list 804 will vary as a function 
of the numbtf of paths that have datagrams pmding for transmission on ou^ut port 
801. If no path is waiting to transmit on output port 801 then the value of both head 

25 pointer 802 and currmt pointer 803 is 0. 

The next datagram,to be sent on output port 801 is determined by current 
pointer 803 which points to the next entry in the transmit list 804 of linked virtual 
path entries. This method of organizing the ouQHit list 804 as a chain of all virtual 
30 paths waiting to transmit on ou^ut port 801 has the effect of giving round robin 
priority to datagram packets waiting to be transmitted from different virtual paths. 

A n^hanism is also provided to send datagram packets fiom selected virtual 
paths at a higher pricnity than other virtual paths. If the priority fiield 322 in the 
35 virtual path record 300 is set, then the value in the priority field 322 indicates the 
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number of pactets to be transmitted from a virtual path before transmitting a packet 
from the next path in the transmit list 804. 

Overall Switch and Network Operati<m 

5 

Referring to Figures 1 through 8, the overall operation of the systrai shall 
now be described. 

For purposes of illustration, assume that client staticm 120 (Figure 1) wants to 
10 send a datagram packet 200 (Hgure 2) to sorver station 126. Datagram 200 will be 
received aa switch input port 401 (Figure 4). Switch hardware gen^-ates virtual path 
index 630 from the datagram destination and source address fields and sends virtual 
path index signals on bus 421 to virtual path cache 415 for lookup. 

IS The virtual path iiKlex in cache 415 will be converted by hash logic 631 to a 

virtual path cache index 632 (Figures 6 and 7) that indexes the four set associative 
virtual path cache 601 tfurough 604 (Figure 6). The virtual path index is further 
compared in parallel against the outputs fr<Hn the foiu: set associate cache 601 
through 604 via the four comparators 611 through 614. Assuming a valid virtual 

20 path tag was found in SRAM cache 601 then comparator 61 1 will indicate a **hit" 
signal on hit/miss wire 635 and enable tri-state buffer 621 to ou^t die virtual path 
record stored in SRAM cache 601 on virtual path record bus 633. 

Switch hardware 409 then forwards and processes the datagram packet 
25 according to the fields of the virtual path recced on bus 633 (Figures 6 and 4). 

If the virtual path cache had not contained a valid virtual path cache entry 
then it would indicate miss on the hitAniss wire 635 (Figures 6 and 4). This causes 
the switch hardware to forward the packet to the CPU 411 via CPU interface 412 for 

30 processing. The controllCT CPU 41 1 then processes the packet in software and sends 
it back to the switch hardware 409 to ou^ut on the appropriate output port if 
controller CPU 411 determines that future datagram packets of this virtual path 
should be processed by switch hardware, ccmtroller CPU 411 then creates a new 
virtual path record for said virtual path and writes it into virtual path cache 415 

35 replacing the least recently used entry if necessary. 
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For example, the invented 



1. supports reliable and efficient data conmiunication in datagram 
networks widiout dropping datagram packets due to lack of network resources; 

2. allows the specification of the attributes of datagram packet flows 
similar to the ci^abilities available in a virtual circuit packet switched network, but 
without the disadvantages of having to set up, maintain, and tear down virtual 
circuits; and 

3. is compatible with existing network protocols and applications, and 
interopeiates with die installed base of datagram netwcMic interfaces. 

The oth^ embodiments of this invention may be obvious to those skilled in 
die art The above description is illustrative only and not linuting. 
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CLAIMS 



Wedaim: 

5 1. In a network device, a method for processing datagram packets wh^ 

the source destination address pair contaimd in the datagram packet determines the 
processing action to be taken. 

2. The method of Claim 1 including the steps of comparing said source 
10 destination address pair contained in the datagram packet to a listing of source 

destination pairs, each source destination pair in the listing being associated with a 
virtual pafli; and 

should a match be found between the source destination address pair 
c<mtained in the datagram packet and a source destination address pair contained in 
IS the listing, selecting the virtual path associated with the source destination address 
pair for transmitting die datagram packet 

3. The method <rf Claim 2 wherein the virtual path for transmission of the 
datagram packet is determined based upon the actual network traffic loading and 

20 the frequency of transmission of datagram packets utilizing the same source- 
destination address pair. 

4. In a netwOTk device, a method fw iHX>cessing datagram packets wh»e 
the source destination address pair contained in the datagram packet determines the 

25 processing actions to be taken based on specifications and state stored in the 
network device. 

5. In a network device Aat processes datagram packets,*a method for 
selecting the processing steps to be applied to a datagram packet, said method 

30 comprising the steps of: 

determining from the source-destination address pair contained in the 
datagram packet a virtual path record craiprising specifications for processing the 
datagram packet; and 

processing the datagram packet according to the processing steps associated 

35 with said virtual paA recmd. 
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6. In a netwofk device, a method for processing datagram packets having 
a source and a destination a^iess comprising the steps of: 

receiving a datagram packet; 

determining from the source-destination address pair contained in.the 
datagram packet a virtual path record c(xnprising specifications for processing die 
datagram packet; and 

processing the datagram packet acc(»ding to the processing steps associated with 
said virtual path record. 

7. The method of processing datagram packets using a iietw(»k device to 
recdve and transmit said datagram pacl^, which conqnises: 

organizing the network device such it can process datagram packets as flows; 

initializing the netwoik device state such that the networic device is ready to 
actually process datagram packets as flows; and 

processing a specific datagram packet as an elem»t of a flow based on the 
source-destination address pair contained in the datagram packet 

8. The method of Claim 7 including: 

modifying the network device state based cm the datagram packets received 
and other processing acticms chosen. 

9. The method of Claim 1 wherein the detmninaticHi of the processing 
action to be takoi is determined by the srarce-destination address pair and Ae type 
field contained in the datagram packet 

10. The n^thod of Claim 1 wherdn the determination of the processing 
action to be taiken is determined by tl^ source-destination address pair and ccmtrats 
of the data field contained in the datagram packet 

11. The method of Claim 1 wherein the datagram packet is one of the 
datagrams defined in the IEEE 802 family of netwoik standards. 

12. The method of Qaim 1 wherein the datagram packet is of the type 
connnonly known as an Ethernet datagram packet and wh^in the source address is 
the Ethernet packet source address field and the destination address is the Ethernet 
packet destination address field. 
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13. The method of Claim 1 wherein the step of processing the datagram 
packets comprises an action chosen from the group consisting of: dropping the 
packet huffering the padcet, forwarding the packet to an ou^ut port» forwarding the 

5 packet to a nmltiplidty of output ports, forwarding a copy of ttie packet to another 
network device, and generating a response back to the sender of the packet 

14. The method (rf Claim 1 wherein processing includes the step of 
forwarding the datagram packet to an (Hi^t port 

10 

15. The method of Qaim 1 wherein processing includes the step of 
forwarding the datagram packet to a multiplicity of output penis. 

16. The method of Qaim 1 wherdn processing includes the st^ of sending 
15 the <iM^grain packet to a controller CPU for further processing. 

17. The method of Claim 1 wherein processing includes the step of 
forwarding a copy of the datagram packet to another network device for processing. 

20 18. The method of.Qaim 1 wherein processing includes the step of 

discarding the datagram packet 

19. The method of Claim 1 wherein processing includes the step of 
notifying the send^ oi the datagram packet that the datagram packet that was sent 

25 has not been accq>ted. 

20. The method of Claim 1 wherein processing includes the step of 
notifying the sender of die datagram padcet diat the datagram packet that was sent 
has not been accepted, wherein said datagram is an Ethernet datagram, and wherein 

30 the response back to the source comi»ises an Ethernet collision signal. 

21. The mediod of Claim 1 wherein the step of f awarding a datagram 
packet includes determining whether the datagram is sent from an authorized source 
to an authorized destinatioiL 

35 
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22. The method Gaim 1 wherein processing includes the step of storing 
the datagram packet in the packet buffer memory means, with separate buffer 
resources provided for each separate virtual path. 

5 23. The method of Claim 1 wherein processing involves determining the 

availability of buffer space on a particular virtual path and not accepting additional 
datagram packets that would exceed the amount of buffer space allowed for this 
virtual path. 

10 24. The method of Claim 1 whmin processing includes ccmmmnicating the 

availability of packet buffer resources available to the upstream sendv of datagram 
packets for this virtual path. 

25. The method of Claim 1 further conqirising allowing the network device 
IS to specify the maximum rate of transmission of a flow of datagrams. 

26. The method of Qaim 1 further comprising a method providing for 
specifying the rate of transmission of one flow of datagrams relative to the rate of 
transmissicm ci other flows of datagrams. 

20 

27. A network device, ccMnprising: 

at least one port for receiving and transmitting datagram packets; 
memory means for buffering said datagram packets; and 
processing means, said device being capable of processing each of said 
25 datagram packets according to instructions stcned in the network device that are 
specific to the source-destination address pair of each datagram pacl^L 

28. A network device as in Claim 27 wherein said at least one port 
coiiq)rises aplurality of ports. 

30 

29. A network device as in Claim 27 wherein said processing defines an 
action to be taken with respect to each datagram packet, said action being chosen 
from the group consisting of: doroi^ing the packet, buffering the packet, forwarding 
the packet to an output port, forwarding die packet to a multiplicity of output ports, 

35 forwarding a copy of the packet to another network device, and generating a 
response back to the send^ of the packet 
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30. The network device of Qaim 27 wherein each datagram packet is one 
of the ^^^fagramg defined in the IEEE 802 family of network standards. 

31. The network device of Claim 27 whorein each datagram packet is of 
the type conmionly known as an Ethernet datagram packet and wherein the source 
address is the Ethernet packet source address field and the destination address is the 
Ediemet packet destinatim address field. 

32. The network device Claim 27 wherein die step of processing each 
datagram packet conqmses a routing action chosen from die group consisting of: 
dr<9ping the packet, buffering the packet, forwarding the packet to an output pwt, 
forwarding the packet to a multiplicity of output ports, forwarding a copy of the 
packet to another network device, and generating a response back to the smder of 
the packet. 

33. The network device of Claim 27, further comprising means for storing 
information in the virtual path record, said information specifying the amount of 
buffer resources provided for each separate fiow of datagram packets. 

34. The network device of Claim 27, further comprising flow control 
means, said flow control means providing the capability to communicate to the 
upstream sender of datagram packets the availability of packet buffer resources 
available. 

35. The netwOTk device of Claim 27, further comprising means for 
allowing,the network system to control the relative rate <^ transmissions among 
nuiltq>Ie flows of datagrams. 

36. The network device of Qaim 27, furth^ ccMipising: 

a virtual path cache memory for providing storage for the most recently used 
virtual path records; 

virtual.patfa cache lodoip means diat allows switch hardware to determine if a 
certain virtual path entry is located in the virtual path cache; 

means for p^orming processing actions specified in die virtual path record if 
a valid virtual path record is found in the virtual path cache memory; and 
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means for processing datagram packets for which no valid entry is found in 
the virtual path cache memory. 



37. The network device of Qaim 27, further comprising a controller CPU 
5 and controller m^ory for processing datagram packets that have no valid virtual 

path record in die virtual path cache, said controller CPU also being able to read and 
write virtual path cache entries. 

38. In a conqmt^ networic syst^ conq>rised of a multiplicity of network 
10 devices, a method for controlling optwcHk c^ieration that specifies fw each network 

device the processing of datagram flows, where each datagram flow is identified by a 
unique source-address destination pair which is contained in the datagram packets 
th^nselves. 

IS 39. The method of Claim 38 wherein specifying the processing includes 

specifying which flows of datagrams receive access to the network. 

40. The method of Qaim 38 wherein specifying; the processing includes 
the step of specifying how to forward flows of datagrams packets. 

20 

41. The method of Claim 38 wherein specifying the processing includes 
die step of specifying how to buffer flows of datagram packets and the buffer 
resources to be used. 

25 42. The n^od of Claim 38 wherein specifying the processing includes 

the stsp of specifying how to schedule the rate of transmission for an individual flow 
of datagram packets. 

43. The method of Claim 38 wherein specifying the processing includes 
30 the step of specifymg how to schedule the rate of transmission among a multiplicity 

of flows of datagram packets. 

44. The method of Claim 38 wherein specifying the processing includes 
monitoring the rate of flows of datagram packets and taking one of several actims 

35 as a fimctim of die flow rate. 
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45. In a datagram packet switched computer networic. comprised of a 
plurality of comiected network devices and host computers, a method for improved 
datagram packet switching, comf^ising the steps of: 

processing datagram packets as separate flows, each flow being idratified by 
S a unique source-destination address pair contained in the datagram itself; and 

maintaining in the netwoik devices infomation that specifies the processing 
of each flow of datagram packets, said information being initially specified during 
network configuration. 

10 46. The method of Qaim 45 wherein processing includes granting access 

to a flow of datagram packets based on availability d buffer resources for such a 
flow. 

47. The method of Claim 45 wherein processing includes specifying tfie 
15 forwarding function of the flow of datagram packets at each netwoik device, which 

is based on die connectivity of the ovenll networic 

48. The method of Claim 45 wherein processing includes specifying the 
scheduling of each flow of datagram packets at each networic device, relative to 

20 resource allocation and traffic congestion at each network node. 

49. In a network device that processes datagram packets, a method for 
forwarding datagram packets, said method conqxrising the steps of: 

determining firom the source-destination address pair contained in each 
25 datagram packet a virtual path record comprising specificatims for forwarding the 
datagram packet; and 

forwarding each datagram packet according to the specificaticms in said 
virtual path record. 

30 50. In a networic device that processes datagram packets, a method for 

buffering datagram packets, said method comprising the steps of: 

determining from the source-destination address pair contained in each 
datagram packet a virtual path record comprising specifications for buffering the 
datagram packet; and 

35 buff^ing the datagram packet according to the specification in said virtual 

path record. 
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